Transcribing Multi-instrument Polyphonic Music with Transformed Eigeninstrument Whole-note Templates

We present a system for the transcription of polyphonic music recordings to recover both the notes played and the instruments responsible for each note. In our framework, the spectrogram of the music is viewed as the superposition of note events, each characterized by an onset time and pitch, an instrument (described by a vector of eigeninstrument weights that combine instrument model bases to match a particular source in the mixture), and per-note transformation parameters that take a durationand decaynormalized note template and extend it to match the actual duration and dynamics of each individual note. Transcription is achieved through an EM-like iterative estimation scheme. Initializing this estimation using a rough separation of sources from a frame-based transcription system gives stable and accurate results that directly describe the audio at a note, rather than a frame, level, with each note attributed to a particular instrument. This approach significantly improves transcription accuracy over a frame-level system, apparently because the transcription constrains each note to obey the dynamics encoded in the templates. Notelevel transcription accuracy on real woodwind excerpts from the MIREX Multiple F0 evaluation improves from 64% (frame-level) to 67%; for the more temporally-structured notes in the RWC piano examples, accuracy improves from 70% to 79%, with dramatic reductions in false alarms.